Take into account changes in field data settings on live indices #4430
Assignees
Labels
Comments
ghost
assigned
jpountz
added a commit
to jpountz/elasticsearch
that referenced
this issue
Dec 12, 2013
…ility to disallow field data loading. This commit changes field data configuration updates so that they are immediately taken into account for loading new segments. The way it works is that field data configuration is now cached separately from the field data cache, meaning that it is now possible to clear the field data configuration from IndexFieldDataService while the cache will stay around. On the next time that Elasticsearch will reload field data configuration, it will check if there is already a cache entry, and reuse it if it exists. To disable field data loading, all that is required is to change the field data format to "none" (supported by all field data types) using the update mapping API. Elasticsearch will then refuse to load field data on any new segment, but field data which has been loaded on the previous segments will remain available. So you need to clear the field data cache in order to reclaim memory (otherwise memory will be reclaimed slower, as segments get merged). Close elastic#4430 Close elastic#4431
jpountz
added a commit
that referenced
this issue
Dec 16, 2013
…ility to disallow field data loading. This commit changes field data configuration updates so that they are immediately taken into account for loading new segments. The way it works is that field data configuration is now cached separately from the field data cache, meaning that it is now possible to clear the field data configuration from IndexFieldDataService while the cache will stay around. On the next time that Elasticsearch will reload field data configuration, it will check if there is already a cache entry, and reuse it if it exists. To disable field data loading, all that is required is to change the field data format to "none" (supported by all field data types) using the update mapping API. Elasticsearch will then refuse to load field data on any new segment, but field data which has been loaded on the previous segments will remain available. So you need to clear the field data cache in order to reclaim memory (otherwise memory will be reclaimed slower, as segments get merged). Close #4430 Close #4431
brusic
pushed a commit
to brusic/elasticsearch
that referenced
this issue
Jan 19, 2014
…ility to disallow field data loading. This commit changes field data configuration updates so that they are immediately taken into account for loading new segments. The way it works is that field data configuration is now cached separately from the field data cache, meaning that it is now possible to clear the field data configuration from IndexFieldDataService while the cache will stay around. On the next time that Elasticsearch will reload field data configuration, it will check if there is already a cache entry, and reuse it if it exists. To disable field data loading, all that is required is to change the field data format to "none" (supported by all field data types) using the update mapping API. Elasticsearch will then refuse to load field data on any new segment, but field data which has been loaded on the previous segments will remain available. So you need to clear the field data cache in order to reclaim memory (otherwise memory will be reclaimed slower, as segments get merged). Close elastic#4430 Close elastic#4431
mute
pushed a commit
to mute/elasticsearch
that referenced
this issue
Jul 29, 2015
…ility to disallow field data loading. This commit changes field data configuration updates so that they are immediately taken into account for loading new segments. The way it works is that field data configuration is now cached separately from the field data cache, meaning that it is now possible to clear the field data configuration from IndexFieldDataService while the cache will stay around. On the next time that Elasticsearch will reload field data configuration, it will check if there is already a cache entry, and reuse it if it exists. To disable field data loading, all that is required is to change the field data format to "none" (supported by all field data types) using the update mapping API. Elasticsearch will then refuse to load field data on any new segment, but field data which has been loaded on the previous segments will remain available. So you need to clear the field data cache in order to reclaim memory (otherwise memory will be reclaimed slower, as segments get merged). Close elastic#4430 Close elastic#4431
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
AbstractFieldMapper.merge has the logic to handle changes of the field data configuration (in particular the format). However, these changes are not going to be taken into account until the field data cache is cleared. It should be possible to update the field data settings on a live index so that field data which has been loaded for the previous segments remains alive while new segments will be loaded using the new field data configuration. As time goes and merges happen, all segments will eventually be using the new format but the transition would be smoother than by clearing the field data cache and regenerating all entries on the next request.
The text was updated successfully, but these errors were encountered: